Computing the power of a t test

Power is defined as the probability of correctly detecting an effect. It is often noted , where the converse, ̄ is the probability of a type-II error (not rejecting H0 when there is an effect). When planning a new experiment, it is generally recommended to have a power of at least .80. Suppose that your design has very little power and suppose further that you found a significant effect. Since you were unlikely to detect it (low power), there are many chances that this effect is truly a type-I error (whose probability, often 5%, is noted ®). Given the fact that you found an effect (and that the presence of an effect is as likely as its absence), the posterior probability that your finding is a type-I error is given by . Hence, with a power of .10 (very low power), it represents a 33% chance of a type-I error rather than a true effect. The power of a 2-group t test depends, as with any statistical test, on three factors: the effect size, the level of significance and the sample size (Cohen, 1992). The larger the expected effect size is, the more powerful the test is likely to be. Likewise, setting the criterion level higher (e.g. .10 instead of .05) will increase power. Figure 1 shows the distribution of the possible results of a t test if there is truly no effect (blue) and if there is a medium effect (red). Increasing the criterion level causes